fix: continue session loop when response is truncated by length (problem with local, smaller LLMs) - #39397
fix: continue session loop when response is truncated by length (problem with local, smaller LLMs)#39397erkkiat wants to merge 2 commits into
Conversation
When a model's finish reason is "length" (hit the output token cap mid-turn) and no tool call was started, the session loop currently ends and goes idle, requiring the user to manually type "continue". This treats "length" like "tool-calls" so the loop keeps going automatically instead.
|
The following comment was made by an LLM, it may be inaccurate: Related PR FoundPR #26167 - This appears to be related as it also handles stream truncations in sessions, though the approach differs (retrying vs continuing). This PR may be worth reviewing to understand any overlapping concerns or previous solutions. The current PR (#39397) is specifically addressing truncations caused by output token limits with local models, while #26167 appears to focus on empty stream truncations. |
|
Automated PR Cleanup Thank you for contributing to opencode. Due to the high volume of PRs from users and AI agents, we periodically close older PRs using automated criteria so maintainers can focus review time on the most active and community-supported contributions. This PR was closed because it matched the following cleanup criteria:
PRs created within the last month are not affected by this cleanup. If you believe this PR was closed incorrectly, or if you are still actively working on it, please leave a comment explaining why it should be reopened. A maintainer can review and reopen it if appropriate. Thanks again for taking the time to contribute. |
1 participant
Issue for this PR
Closes #17471
Type of change
What does this PR do?
When using local models such as qwen3.6:35b, the context window limitations will break the flow on a regular basis. When a model's finish reason is "length" (hit the output token cap mid-turn) and no tool call was started, the session loop currently ends and goes idle, requiring the user to manually type "continue". This treats "length" like "tool-calls" so the loop keeps going automatically instead. The fix suggested in #17471 can work, but here is an alternate, perhaps cleaner solution.
How did you verify your code works?
I have used a version with this fix for a couple of weeks, and it works continuously without requiring manual intervention. I want to share a working solution.
Screenshots / recordings
No UI changes.
Checklist